Understanding the “Sum of Subtest to Overall Score Discrepancy” on the Maintenance of Certification-Family Practice Examination

When high-stakes examinations, such as the American Board of Family Medicine’s (ABFM’s) Maintenance of Certification-in Family Practice (MCFP) examination, are administered, candidates and diplomates are keenly interested in the accuracy of their test scores, especially when their scores are close to, but below, the pass/fail cutpoint. In some instances, candidates will attempt to reverse engineer their scores using the information provided on the score report in an effort to verify that the “weighted sum of the subtest scores” is congruent with the overall test score. Any discrepancy might become alarming to the candidate, providing a seemingly legitimate reason to believe the overall score was inaccurate, thus prompting a phone call to the ABFM for further investigation and clarification. Historically, such a mistake in scoring has never been found; however, a statistical phenomenon that we will describe below could make it appear so. We would like to explain this phenomenon so that examinees who attempt to reverse engineer their score reports will better understand the “sum of subtest to overall score discrepancy” phenomenon.